Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add transfer-functions and hdr-metadata to video-capabilities. #300

Merged
merged 4 commits into from
Nov 7, 2022

Conversation

markafoltz
Copy link
Contributor

@markafoltz markafoltz commented Sep 9, 2022

Addresses Issue #194: [Remote Playback] Capabilities for HDR rendering and display

This adds two fields to the receive-video-capability message that allows the receiver to list the HDR metadata types and transfer functions that the media receiver can accept.

The values are used by reference from the corresponding enums in the Media Capabilities API. This was discussed, briefly, in the minutes of the joint SSWG/Media WG meeting:

https://www.w3.org/2022/01/11-mediawg-minutes.html#t01


Preview | Diff

@markafoltz
Copy link
Contributor Author

One question that could use further input is whether the two new values are independent, i.e. if you have:

transfer-functions: [  "p3", "rec2020"  ]
hdr-metadata: [ "smpteSt2086", "smpteSt2094-10" ]

Is it always true that all four combinations are possible? If not, then we might have to adjust the data model to be a list of transfer-function and hdr-metadata pairs.

Probably requires looking at some HDR displays to see what the constraints are in the real world.

@markafoltz
Copy link
Contributor Author

After reviewing some of the relevant documentation around the three linked fields color-gamuts, transfer-functions and hdr-metadata, here is what I think makes sense for the protocol.

  • The color gamut values are inclusive by definition: rec2020 includes p3, which includes srgb. We could change color-gamuts from a list to a single field, as long as we made it clear that the agent receiving the color-gamut applied the inclusion logic. I will file an issue for this, but I don't feel strongly that this requires a change to the protocol.

  • Some hdr-metadata and transfer-functions values don't make sense. For example HLG does not require a separate metadata format, so a media receiver that understood both e.g. HDR10+ and HLG via one codec would have to report them as two separate encoding types with all other fields duplicated:

Encoding 1:

<repeat everything else>
hdr-metadata: []
transfer-functions: ['hlg']
<repeat everything else>

Encoding 2:

<repeat everything else>
hdr-metadata: ['smpteSt2094-40']
transfer-functions: ['pq']
<repeat everything else>

For this reason I think we should report the HDR capabilities as a list of pairs, so this can be conveyed in one encoding:

<snip>
hdr-formats: [
  { transfer-function: 'hlg' },
  { transfer-function: 'pq', hdr-metadata: 'smpteSt2094-40' }
]
<snip>

meaning that encoding could convey either HLG or HDR10+. For completeness, here is what it would look like with smpteSt2094-10 included, which can support either HLG or PQ [1].

<snip>
hdr-formats: [
  { transfer-function: 'hlg' },  // HLG with no metadata
  { transfer-function: 'hlg', hdr-metadata: 'smpteSt2094-10' }  // 2094-10 with HLG. [2]
  { transfer-function: 'pq', hdr-metadata: 'smpteSt2094-10' }   // 2094-10 with PQ
]
<snip>

Note, per the issue linked below, Dolby has registered a number of MIME types to represent their proprietary HDR metadata encodings, so remote playback capability detection for these formats would be handled by the remote-playback-source message, which accepts full MIME types.

[1] w3c/media-capabilities#136
[2] Unclear if if this happens in practice as HLG support may be a proprietary profile of Dolby Vision and not part of smpteSt2094-10.

@markafoltz
Copy link
Contributor Author

@chrisn, we have resolved the issues preventing PRs from landing on the OSP spec. Feel free to take a look.

@chrisn
Copy link
Member

chrisn commented Oct 19, 2022

Thank you, somehow I missed your previous comment here, I'll take a look.

@chrisn
Copy link
Member

chrisn commented Oct 28, 2022

We could change color-gamuts from a list to a single field, as long as we made it clear that the agent receiving the color-gamut applied the inclusion logic. I will file an issue for this, but I don't feel strongly that this requires a change to the protocol.

I agree.

For this reason I think we should report the HDR capabilities as a list of pairs

This sounds good.

Dolby has registered a number of MIME types to represent their proprietary HDR metadata encodings, so remote playback capability detection for these formats would be handled by the remote-playback-source message, which accepts full MIME types.

I see. If we need to we can follow this up with them via the new A/V Media Formats CG, which started at TPAC 2022.

@markafoltz
Copy link
Contributor Author

We could change color-gamuts from a list to a single field, as long as we made it clear that the agent receiving the color-gamut applied the inclusion logic. I will file an issue for this, but I don't feel strongly that this requires a change to the protocol.

I agree.

Filed #305

@markafoltz
Copy link
Contributor Author

For this reason I think we should report the HDR capabilities as a list of pairs

This sounds good.

Ok, updated PR to reflect this change.

@markafoltz
Copy link
Contributor Author

Feel free to review the current PR @chrisn with the changes I proposed, however, the spec generation is currently broken again so you won't be able to see an HTML preview yet. ☹️

@chrisn
Copy link
Member

chrisn commented Oct 31, 2022

Thanks. I've asked a colleague for advice on this part:

[2] Unclear if this happens in practice as HLG support may be a proprietary profile of Dolby Vision and not part of smpteSt2094-10.

@markafoltz
Copy link
Contributor Author

Hi @chrisn, did you have any additional feedback on the PR?

Regarding:

[2] Unclear if this happens in practice as HLG support may be a proprietary profile of Dolby Vision and not part of smpteSt2094-10.

Would definitely appreciate any feedback on this. However, the protocol proposed in the PR should support this use case if it is necessary.

Copy link
Member

@chrisn chrisn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is good, thanks!

@markafoltz markafoltz merged commit 246bc07 into main Nov 7, 2022
@markafoltz
Copy link
Contributor Author

Thanks!

@markafoltz markafoltz deleted the issue-194-hdr branch November 7, 2022 20:00
github-actions bot added a commit that referenced this pull request Nov 7, 2022
SHA: 246bc07
Reason: push, by mfoltzgoogle

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants